Statistical Mechanics of MAP Estimation: General Replica Ansatz

The large-system performance of maximum-a-posterior estimation is studied considering a general distortion function when the observation vector is received through a linear system with additive white Gaussian noise. The analysis considers the system matrix to be chosen from a large class of random ensembles. We take a statistical mechanical approach by introducing a spin glass corresponding to the estimator, and employing the replica method for the large-system analysis. In contrast to earlier replica based studies, our analysis evaluates the general replica ansatz of the corresponding spin glass and determines the asymptotic distortion of the estimator for any structure of the replica correlation matrix. Consequently, the replica symmetric as well as the replica symmetry breaking ansatz with b steps of breaking is deduced from the given general replica ansatz. The generality of our distortion function lets us derive a more general form of the maximum-a-posterior decoupling principle. Based on the general replica ansatz, we show that for any structure of the replica correlation matrix, the vector-valued system decouples into a bank of equivalent decoupled linear systems followed by maximum-a-posterior estimators. The structure of the decoupled linear system is further studied under both the replica symmetry and the replica symmetry breaking assumptions. For b steps of symmetry breaking, the decoupled system is found to be an additive system with a noise term given as the sum of an independent Gaussian random variable with b correlated impairment terms. The general decoupling property of the maximum-a-posterior estimator leads to the idea of a replica simulator which represents the replica ansatz through the state evolution of a transition system described by its corresponding decoupled system. As an application of our study, we investigate large compressive sensing systems by considering the lp norm minimization recovery schemes. Our numerical investigations show that the replica symmetric predicted performance of the l0 norm recovery scheme fails as the compression rate grows, and therefore, the replica symmetry breaking ansätze are needed in order to assess the performance precisely. Index Terms Maximum-a-posterior estimation, linear vector channel, decoupling principle, equivalent single-user system, compressive sensing, zero norm, replica method, statistical physics, replica symmetry breaking, replica simulator This work was supported by the German Research Foundation, Deutsche Forschungsgemeinschaft (DFG), under Grant No. MU 3735/2-1. Ali Bereyhi and Ralf R. Müller are with the Institute for Digital Communications (IDC), Friedrich Alexander University of Erlangen-Nürnberg (FAU), Konrad-Zuse-Straße 5, 91052, Erlangen, Bavaria, Germany (e-mails: [email protected], [email protected]). Hermann Schulz-Baldes is with the Department of Mathematics, FAU, Cauerstraße 11, 91058, Erlangen, Bavaria, Germany (e-mail: schuba@ mi.uni-erlangen.de).